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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 17 April 2007 . 
2a)IEl This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) K Claim(s) 1-28 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) Q The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S:C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



1. 



This Office Action is in response to the Amendments/Arguments submitted 17 April 2007. 



2. 



Claims 1-28 have been presented for examination. Claims 21-28 are newly added. 



Response to Arguments 



3. 



Applicant's arguments with respect to claims 1 -20 have been considered but are moot in view of 



the new ground(s) of rejection. 



Claim Rejections - 35 USC 6 102 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis 
for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (l) an application for patent, published under section 122(b), by another 
filed in the United States before the invention by the applicant for patent or (2) a patent granted on an 
application for patent by another filed in the United States before the invention by the applicant for patent, 
except that an international application filed under the treaty defined in section 35 1(a) shall have the effects 
for purposes of this subsection of an application filed in the United States only if the international 
application designated the United States and was published under Article 21(2) of such treaty in the English 



4. Claims 1, 3-5, 7-9, 10-12, 14-15, 17-18, 21-25 and 27-28 are rejected under 35 U.S.C. 102(e) 
as being clearly anticipated by Ralston (US Pub. No. 2003/0014212). 

Regarding claims 1 and 27: 

Ralston discloses a visualization system for a computer system including: 



a. a positioning portion (figure 4a head position 405 renderer 400; [0057]) configured to 
determine a position of a viewer with respect to a real world (Figure 7; [0068]) and a 
position of the viewer with respect to a virtual world (figure 8; [0070]), the positioning 
portion configured to allow the viewer to interact with the virtual world (figure 14; 
[0076]) 



language 
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b. a modeling portion configured to specify the virtual world in response to a model of the 
virtual world (figure 1 Tenderer 400; [0057]) 

c. a model specification portion configured to specify a representation of satellite status data 
in response to the position of the viewer with respect to the virtual world and in response 
to the satellite status data (figure 25; [0090]-[0091]) 

d. an output portion configured to provide an image of the virtual super-imposed on an 
image of the real world, the image of the virtual world including the representation of the 
satellite status data to the viewer in response to the position of the viewer with respect to 
the virtual world (figure 25; [0090]-[0091]; figure 27 output to 465) 

Regarding claim 27, the "object" in this claim is equivalent to the "satellite" in claim 1 . 

Regarding claims 3 and 10: 

Ralston discloses the visualization system of claim 1 wherein the model of the virtual world is a 
multi-dimensional model of the virtual world ([0100]), and the output portion in a multi-dimensional 
output portion ([0062] real-world output presented in form of see through glasses, which is 3D). 

Regarding claims 4, 11, 17 and 28: 

Ralston discloses the visualization system of claim 1 wherein the representation of the satellite 
data comprises a representation of the current position (figure 25; [0038]). 

Regarding claims 5, 12 and 18: 

Ralston discloses the visualization system of claim 1 wherein the three-dimensional output 
portion comprises a heads-up pair of glasses (figure 4B headset 465; [0058]; figure 9; [0071]). 
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Regarding claims 7 and 14: 

Ralston discloses the visualization system of claim 5 wherein the heads-up pair of glasses area 
also configured to allow the viewer to view the image of the virtual world super-imposed on the image of 
the real world ([0057] real world field of view augmented with virtual objects). 

Regarding claim 8: 

Ralston discloses a method for visualization of augmented reality, the method comprising: 

a. determine a position of a viewer with respect to a real world (figure 7; [0068]) and a 
position of the viewer with respect to a virtual world (figure 8; [0070]), the positioning 
portion configured to allow the viewer to interact with the virtual world (figure 14; 
[0076]) 

b. determining a model of the virtual world (figure 7 steps 720 and 725). 

c. specify a representation of satellite status data in response to the position of the viewer 
with respect to the virtual world and in response to the satellite status data (figure 25; 
[0090]-[0091]) 

d. displaying to the viewer a representation of the virtual world super-imposed on a 
representation of the real world ([0057] real world field of view augmented with 
virtual objects), the representation of the virtual world including the representation of 
the satellite status data in response to the position of the viewer with respect to the virtual 
world (figure 25; [0090[-[0091]). 



Regarding claim 15: 

Ralston discloses a visualization method for a computer system comprising: 
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a. displaying to a viewer a representation of a real world overlaid with a representation of a 
virtual world ([0057] real world field of view augmented with virtual objects), the 
representation of the virtual world including the representation of the satellite status data 
(figure 25; [0090]-[0091]). 

b. wherein the representation of the virtual world is determined in response to the model of 
the virtual world (figure 7 steps 720 and 725) and in response to a position of the viewer 
with respect to the virtual world (figure 7; [0068]; figure 8; [0070]). 

c. wherein the representation of the satellite status data is determined in response to satellite 
status data, and in response to a position of the viewer with respect to the virtual world 
(figure 25; [0090]-[0091]). 

d. wherein the viewer is allowed to interact with the virtual world (figure 14; [0076]) 

Regarding claims 21-23: 

Ralston discloses providing the representation of the real world and the virtual world (and super- 
imposing the virtual world on the real world) in real time ([0055]; [0057]). 

Regarding claim 24: 

Ralston discloses allowing the viewer to select a satellite or geographic area ([0091]). 
Regarding claim 25: 

Ralston discloses allowing the viewer to directly select and manipulate objects in the virtual 
world without using a mouse ([0061]). 
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Claim Rejections - 35 USC S 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

. (a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior 
art are such that the subject matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 



The factual inquiries set forth in Graham \.John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), 
that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are 
summarized as follows: 

1. Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or 
nonobviousness. 

This application currently names joint inventors. In considering patentability of the claims under 
35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly 
owned at the time any inventions covered therein were made absent any evidence to the contrary. 
Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of 
each claim that was not commonly owned at the time a later invention was made in order for the examiner 
to consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) prior art under 
35 U.S.C. 103(a). 



5. Claims 2, 6, 9, 13, 16, 19, 20 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ralston (US Pub. No. 2003/0014212) in view of Kato ("Virtual Object Manipulation on a 
Table-Top AR Environment", 2000). 



Regarding claims 2, 9 and 16: 

Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato 
teaches an augmented reality system including: 
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a. an image acquisition source configured to capture at least an image comprising an image 
of the real world (Kato: section 3 4 th paragraph "users... see the real world through 
the video camera; section 4.1), and an image of at least a pre-determined marker 
positioned in the real world (Kato: section 3 4 ,h -5 th paragraphs: real cards labeled 
with tracking markers, and computer vision techniques are used to find the tracking 
mark) 

b. an image processing portion configured to determine the position of the viewer with 
respect to the read world in response to the image of the pre-determined marker (s Kato: 
section 3 4 th paragraph determine the exact pose of the head mounted camera 
relative to the tracker; section 4.1) 

c. virtual positioning portion configured to translate the position of the viewer in the real 
world to the position of the viewer in the virtual world (Kato: section 3 4 th paragraph 
once the position of the real camera is known, virtual image can be exactly overlaid 
on it; section 4.1) 

At the time of the invention, it would have been obvious to one of ordinary skill in the arts to 
combine the teachings of Ralston and Kato because the interface of Kato provides accurate tracking and 
registration techniques and an intuitive and useful interface (Kato: abstract). 

Regarding claims 6 and 13: 

Kato teaches the visualization system of claim 2 wherein the image acquisition source (a video 
camera) is physically coupled to the heads-up pair Of glasses (section 3 1 st paragraph "people... wear 
Olympus HMDs with cameras attached"). 



Regarding claim 19: 
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Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato 
teaches an augmented reality system including displaying to the viewer a portion of the virtual selected 
by the viewer wherein the viewer selection is determined in response to a position of the a viewer- 
controlled marker with respect to the virtual world, wherein the marker is positioned in the real world 
(Kato: section 3). At the time of the invention, it would have been obvious to one of ordinary skill in the 
arts to combine the teachings of Ralston and Kato because the interface of Kato provides accurate 
tracking and registration techniques and an intuitive and useful interface (Kato: abstract). 

Regarding claim 20: 

Kato discloses the visualization method of claim 19, wherein the step of displaying to the viewer 
the portion of the virtual world selected by the viewer comprises overlaying an icon over the portion of 
the virtual world displayed to the viewer (Kato: section 3). 

Regarding claim 26: 

Ralston does not explicitly disclose the use of pre-determined markers in the real world. Kato 
teaches an augmented reality system including a marker positioned in the real world, the marker is static 
or placed upon a paddle that included a pre-defined visual marker in the real world, and the paddle is 
capable of being moved around the real world (Kato: sections 3 and 4). At the time of the invention, it 
would have been obvious to one of ordinary skill in the arts to combine the teachings of Ralston and Kato 
because the interface of Kato provides accurate tracking and registration techniques and an intuitive and 
useful interface (Kato: abstract). 
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Conclusion 

6. Examiner's Remarks: Examiner has cited particular columns and line numbers in the references 
applied to the claims above for the convenience of the applicant. Although the specified citations are 
representative of the teachings of the art and are applied to specific limitations within the individual 
claim, other passages and figures may apply as well. It is respectfully requested from the applicant in 
preparing responses, to fully consider the references in their entirety as potentially teaching all or part of 
the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the 
Examiner. In the case of amending the claimed invention, Applicant is respectfully requested to indicate 
the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also 
to verify and ascertain the metes and bounds of the claimed invention. 

7. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing 
date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH 
shortened statutory period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later than SIX 
MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Shambhavi Patel whose telephone number is (571) 272-5877. The examiner can normally 
be reached on Monday-Friday, 8:00 am - 4:30 pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Kamini Shah can be reached on (571) 272-2279. The fax phone number for the organization where this 
application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained 
from either Private PAIR or Public PAIR. Status information for unpublished applications is available 
through Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



SKP 




